Repository navigation
Cut worker and result response bytes: capabilities digest, history-free completed result (#352) - #354
Merged
Merged
Conversation
…ee result (#352) Worker-plane requests opt in to the server capabilities digest (durable-workflow/server#1262): they send X-Durable-Workflow-Server-Capabilities-Digest with the digest held when the request is sent (or "none"), and the answer, success or error body and negotiated poll envelopes included, is resolved against that captured pair up to three levels deep. A tagged full copy is learned for later requests and its digest member stripped; an unknown digest raises ServerCapabilitiesDigestError. A request in flight across a rolling deploy restores the object it was sent with. get_result returns a completed run's output from the status read's output_envelope when it names the same run, skipping the history read; non-success outcomes and envelope-less answers keep the history path. gzip needs no change: httpx advertises and decodes it (test added). Co-Authored-By: Claude Opus 5.5 <[email protected]>
httpx decodes gzip transparently, so a small compressed answer could inflate without limit (a reviewer expanded 1.5 MB of gzip to 3.2 GB in other SDKs). A response hook wraps each body stream and counts the bytes it decodes to, inflating at most 1 MiB at a time, and raises the new ResponseBodyTooLarge (not retried) before httpx decodes past the bound. Configurable with Client(max_response_body_bytes=...), also on the sync client. The client now asks only for gzip; another response Content-Encoding is refused. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Keep both error additions: ResponseBodyTooLarge and ServerCapabilitiesDigestError beside main's ActivityWorkerLost (and its ActivityLeaseAuthorityLost alias). Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
rmcdaniel
added a commit
that referenced
this pull request
Nov 20, 2026
Keeps both: this branch's transient retries, committed-completion rule and outage logs, and main's decoded response cap and continue-as-new result. Only CHANGELOG.md conflicted; both entries are kept. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Items 1–3 of #352 (item 4 is #353). Needs no Server change. With durable-workflow/server#1261 (compression) and durable-workflow/server#1262 (capabilities digest), both merged and shipping in 3.17.0, this brings the Python SDK in line with durable-workflow/sdk-php#446.
Changes
Capabilities digest (
src/durable_workflow/_capabilities_digest.py, contractdocs/contracts/server-capabilities-digest.mdon server main):_request(..., worker=True)) sendX-Durable-Workflow-Server-Capabilities-Digest: <digest>|none. Discovery and control-plane requests do not.digestmember) is learned for later requests, and the member is stripped before callers see it. A digest string that names neither the captured pair nor a tagged copy in the same answer raises the newServerCapabilitiesDigestError.History-free result: for a
completedrun,get_resultreturns the output thatdescribe_workflowalready decoded from the status read'soutput_envelope, when the status read names the same run. It then skips the history read. The history path stays for failed, cancelled, terminated and timed-out runs, for a status read withoutoutput_envelope, and for a handle pinned to a different run.WorkflowExecutiongains a private, non-init, non-compared flag that records whetheroutputcame from the envelope.gzip and a bound on decoded bodies: httpx decodes gzip transparently, so a small compressed answer could inflate without limit. A response hook now wraps each body stream and counts the bytes it decodes to, inflating at most 1 MiB at a time.
Client(max_response_body_bytes=...)(128 MiB by default, also on the sync client), it raises the newResponseBodyTooLarge, which is not retried, before httpx decodes past the bound.gzip. Any other responseContent-Encodingis refused.tests/test_response_body_limit.py):Tests
tests/test_capabilities_digest.py:noneuntil a tagged copy is learneddigestmember is strippedClient._requestwith a mock server that elides like server#1262:WorkflowFailedLocally (Python 3.14, httpx 0.28.1),
pytest tests/ -m "not integration"gives 7794 passed and 3 skipped, and ruff, mypy and the regression-corpus guard are clean.Measurement
Setup:
43ad8d8f(3.17.0-main43ad8d8f, with server#1261 and server#1262), on SQLite, capped at 1 CPU / 1 GiB, with default compression on./home/lab4/fleet/tmp/general-egress-py/scripts/measure.py). It counts wire bytes per response with httpx hooks. The first 3 s of worker start-up (registration, first tagged copy) are excluded from both runs.0d5283cWhere the worker savings come from: with the digest, each worker answer carries the 73-byte digest string in place of the ~16.9 KB capabilities object. For example, a workflow-task completion drops from 11,025 to 676 bytes, and the activity
renew-leaseanswer from 5,676 to 876. Several answers are now small enough that the Server no longer compresses them.Earlier baseline: for comparison with #352, the same workflow on stock 3.15.0 without compression moved 193,799 bytes per workflow.
🤖 Generated with Claude Code